Skip to content

Run KubeVirt integration tests on GitHub Actions#287

Merged
Jakob-Naucke merged 5 commits into
trusted-execution-clusters:mainfrom
Jakob-Naucke:gha-kubevirt
Jun 26, 2026
Merged

Run KubeVirt integration tests on GitHub Actions#287
Jakob-Naucke merged 5 commits into
trusted-execution-clusters:mainfrom
Jakob-Naucke:gha-kubevirt

Conversation

@Jakob-Naucke

@Jakob-Naucke Jakob-Naucke commented Jun 23, 2026

Copy link
Copy Markdown
Member

Summary by Sourcery

Refactor integration tests and supporting utilities to use Kubernetes await-based conditions and stricter cleanup, enhance operator installation error handling and logging, add configuration for KubeVirt VM CPU requirements, and introduce a GitHub Actions workflow and supporting Go module to run KubeVirt-based integration tests in CI.

Enhancements:

  • Replace custom polling logic in tests and utilities with kube runtime await_condition and timeout-based waiting helpers for resource readiness and deletion.
  • Strengthen test cleanup by generically verifying absence of leftover AttestationKey, ApprovedImage, and Machine resources before tearing down namespaces.
  • Improve operator reconcile behavior by factoring component installation into a dedicated helper that returns structured errors and triggers requeue on installation failures, with clearer logging and context.
  • Parameterize KubeVirt VM CPU resource requests via an environment variable to better support low-core CI environments.
  • Update build tooling to derive the kopium version from cargo metadata and allow overriding the operator image tag, and expand REUSE metadata coverage for new tooling files.

CI:

  • Add a GitHub Actions workflow to build images, provision a KinD+KubeVirt environment, and execute KubeVirt-based integration tests on pull requests from trusted or explicitly approved contributors.
  • Introduce a dedicated Go module for tracking the virtctl (KubeVirt CLI) version and integrate it into Dependabot configuration for automated dependency updates.
  • Download and install virtctl in CI based on the version pinned in the new Go module, ensuring compatibility with the KubeVirt version used in tests.

Documentation:

  • Document new test configuration options for adjusting KubeVirt CPU resource requests and timeouts in the integration test README.

Tests:

  • Simplify and harden integration tests by reusing shared condition helpers, updating resource wait helpers, and ensuring robust lifecycle checks for AttestationKey, ConfigMap, ApprovedImage, and namespace resources.

@openshift-ci

openshift-ci Bot commented Jun 23, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@sourcery-ai

sourcery-ai Bot commented Jun 23, 2026

Copy link
Copy Markdown

Reviewer's Guide

Refactors integration tests and operator installation flow to support running KubeVirt-based integration tests in GitHub Actions, replacing ad‑hoc polling with kube runtime await_condition/timeouts, tightening test cleanup, and adding a dedicated Go module and CI workflow for virtctl/KubeVirt setup.

Sequence diagram for updated operator reconcile installation flow

sequenceDiagram
    participant Reconcile as reconcile
    participant InstallComponents as install_components
    participant ReferenceValues as reference_values::adopt_approved_images

    Reconcile->>InstallComponents: install_components(client, cluster)
    alt [component installation fails]
        InstallComponents-->>Reconcile: Err(e)
        Reconcile-->>Reconcile: Action::requeue(Duration::from_secs(60))
    else [component installation succeeds]
        InstallComponents-->>Reconcile: Ok(())
        Reconcile->>ReferenceValues: adopt_approved_images(client, cluster)
        ReferenceValues-->>Reconcile: Ok(())
        Reconcile-->>Reconcile: Action::await_change()
    end
Loading

File-Level Changes

Change Details Files
Replace custom Poller-based wait logic in Rust tests with kube::runtime::wait::await_condition plus tokio::time::timeout and simplify resource created/deleted helpers.
  • Introduce small, strongly-typed predicate helpers (e.g., ak_approved, populated, chk_removed) that drive await_condition calls for AttestationKey, ConfigMap, ApprovedImage, and Secret states.
  • Refactor wait_for_resource_created and wait_for_resource_deleted to be generic helpers using await_condition and timeout instead of custom polling loops and interval parameters.
  • Remove deployment-specific polling helper in favor of a generic await_condition-based deployment readiness check using DeploymentStatus conditions.
  • Update attestation, trusted_execution_cluster, and KubeVirt VM lifecycle tests to use the new helpers for readiness, deletion, and VM running state.
tests/trusted_execution_cluster.rs
tests/attestation.rs
test_utils/src/lib.rs
test_utils/src/virt/kubevirt.rs
Tighten test cleanup to fail if TEC-related resources are left behind after tests and simplify machine deletion handling.
  • Change TestContext::cleanup to poll until no AttestationKey, ApprovedImage, or Machine resources remain, failing the test if any are still present after a timeout.
  • Add a generic check_no_resources helper that lists namespaced resources and errors if any items are found.
  • Remove the bespoke delete_machines helper and rely on the generic cleanup path for Machines.
  • Standardize namespace deletion to use the new wait_for_resource_deleted signature without explicit intervals.
test_utils/src/lib.rs
Make operator reconciliation fail-fast on component installation errors and requeue instead of proceeding with a partially installed cluster, while adding structured error contexts.
  • Introduce an install_components helper that orchestrates trustee configuration, register server, and attestation key register installation and propagates errors.
  • Wrap each installer (trustee configuration, register server, attestation key register) in anyhow::Context with specific messages so reconcile logs include detailed failure causes.
  • Change reconcile to call install_components and, on error, log a warning and return a requeue Action instead of silently continuing.
  • Update the reconcile unit test stubbed HTTP responder to return different dummy resource types (ConfigMaps, Services, Deployments) for each POST to keep tests passing under the new installer behavior.
operator/src/main.rs
Add a dedicated virtctl Go module to control the virtctl/kubevirt client version used by tests and hook it into tooling and licensing metadata.
  • Create tools/virtctl/go.mod and go.sum defining a module that pins kubevirt.io/kubevirt and associated indirect dependencies, with explicit replace directives to match the Kubernetes/KubeVirt stack.
  • Add an empty tools/virtctl/tools.go to make the module buildable as a tool package.
  • Extend Dependabot configuration to manage Go module updates in tools/virtctl and include the new go.sum in REUSE licensing metadata.
tools/virtctl/go.mod
tools/virtctl/go.sum
tools/virtctl/tools.go
.github/dependabot.yml
REUSE.toml
Introduce a GitHub Actions workflow that provisions a KinD cluster with KVM-enabled KubeVirt, builds/pushes operator images, and runs the Rust integration tests.
  • Add .github/workflows/integration-tests.yml that sets up Rust and Go toolchains, installs KinD, downloads virtctl based on the pinned kubevirt module version, creates a KinD cluster, builds and pushes images to a local registry, installs KubeVirt, and runs integration tests gated by author_association/ok-to-test label.
  • Configure workflow-level environment variables for container runtime, registry, and TEST_KV_CPU_RESOURCE_REQ to make KubeVirt tests usable in CI.
  • Enable concurrency cancellation keyed to the PR to avoid wasting CI resources on superseded revisions.
.github/workflows/integration-tests.yml
General build/test tooling tweaks for better configurability and CI friendliness.
  • Make OPERATOR_IMAGE overridable in the Makefile so CI can point tests at a local registry image.
  • Change KOPIUM_VERSION discovery to use cargo metadata via jq instead of grepping Cargo.toml so version resolution is more robust.
  • Document TEST_KV_CPU_RESOURCE_REQ in tests/README.md as a way to reduce KubeVirt VM CPU requests on constrained systems.
  • Remove a stale comment about tracking the k8s version in Cargo.toml’s k8s-openapi dependency.
Makefile
tests/README.md
Cargo.toml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@Jakob-Naucke

Copy link
Copy Markdown
Member Author

/ok-to-test

@Jakob-Naucke

Copy link
Copy Markdown
Member Author

first pass, 3 passes in a row & I'm happy

@Jakob-Naucke Jakob-Naucke force-pushed the gha-kubevirt branch 2 times, most recently from b95e93a to ffd95fe Compare June 26, 2026 08:32
@Jakob-Naucke

Copy link
Copy Markdown
Member Author

3 passes, removing debug info

@Jakob-Naucke Jakob-Naucke marked this pull request as ready for review June 26, 2026 10:47
@Jakob-Naucke Jakob-Naucke requested a review from alicefr June 26, 2026 10:47
@Jakob-Naucke

Copy link
Copy Markdown
Member Author

Opening for review. To reviewers: This reuses some commits identified in #285, but I'm still opening it now because the feedback loop for making KubeVirt-on-KinD-GHA work is much quicker than Azure-on-OpenShift-CI.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • In tools/virtctl/go.mod the go 1.25.0 directive targets a Go version that is not yet generally available; consider aligning this with a currently supported Go release to avoid go list/tooling failures when working in that module.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `tools/virtctl/go.mod` the `go 1.25.0` directive targets a Go version that is not yet generally available; consider aligning this with a currently supported Go release to avoid `go list`/tooling failures when working in that module.

## Individual Comments

### Comment 1
<location path="Makefile" line_range="21" />
<code_context>
 YQ_VERSION ?= $(shell go list -m -f '{{.Version}}' github.com/mikefarah/yq/v4)
 YQ ?= $(LOCALBIN)/yq-$(YQ_VERSION)
-KOPIUM_VERSION ?= $(shell grep kopium lib/Cargo.toml | sed -E 's/.*"(.*)"/\1/')
+KOPIUM_VERSION ?= $(shell cargo metadata --format-version 1 | jq -r '.resolve.nodes[] | select(.deps[]?.name == "kopium") | .deps[] | select(.name == "kopium") | .pkg | split("@")[1]')
 KOPIUM ?= $(LOCALBIN)/kopium-$(KOPIUM_VERSION)

</code_context>
<issue_to_address>
**suggestion (performance):** Using `cargo metadata` + `jq` here introduces extra tooling requirements and may slow down repeated Make invocations.

This approach is nicer than grepping Cargo.toml, but it does mean:

- `jq` must be installed, and
- `cargo metadata` runs on every evaluation, which can noticeably slow CI or frequent `make` runs.

Consider either caching `KOPIUM_VERSION` via a helper script/Make target so it’s computed once and reused, or providing a simpler fallback (e.g., the previous approach) when `jq`/`cargo` aren’t available to keep `make` usable in minimal environments.

```suggestion
KOPIUM_VERSION := $(shell cargo metadata --format-version 1 2>/dev/null \
	| jq -r '.resolve.nodes[] | select(.deps[]?.name == "kopium") | .deps[] | select(.name == "kopium") | .pkg | split("@")[1]' 2>/dev/null \
	|| grep kopium lib/Cargo.toml | sed -E 's/.*"(.*)"/\1/')
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread Makefile
YQ_VERSION ?= $(shell go list -m -f '{{.Version}}' github.com/mikefarah/yq/v4)
YQ ?= $(LOCALBIN)/yq-$(YQ_VERSION)
KOPIUM_VERSION ?= $(shell grep kopium lib/Cargo.toml | sed -E 's/.*"(.*)"/\1/')
KOPIUM_VERSION ?= $(shell cargo metadata --format-version 1 | jq -r '.resolve.nodes[] | select(.deps[]?.name == "kopium") | .deps[] | select(.name == "kopium") | .pkg | split("@")[1]')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (performance): Using cargo metadata + jq here introduces extra tooling requirements and may slow down repeated Make invocations.

This approach is nicer than grepping Cargo.toml, but it does mean:

  • jq must be installed, and
  • cargo metadata runs on every evaluation, which can noticeably slow CI or frequent make runs.

Consider either caching KOPIUM_VERSION via a helper script/Make target so it’s computed once and reused, or providing a simpler fallback (e.g., the previous approach) when jq/cargo aren’t available to keep make usable in minimal environments.

Suggested change
KOPIUM_VERSION ?= $(shell cargo metadata --format-version 1 | jq -r '.resolve.nodes[] | select(.deps[]?.name == "kopium") | .deps[] | select(.name == "kopium") | .pkg | split("@")[1]')
KOPIUM_VERSION := $(shell cargo metadata --format-version 1 2>/dev/null \
| jq -r '.resolve.nodes[] | select(.deps[]?.name == "kopium") | .deps[] | select(.name == "kopium") | .pkg | split("@")[1]' 2>/dev/null \
|| grep kopium lib/Cargo.toml | sed -E 's/.*"(.*)"/\1/')

for CI installation & dependabot autoupdates

Signed-off-by: Jakob Naucke <jnaucke@redhat.com>
This reverts commit cc6bf85.

Signed-off-by: Jakob Naucke <jnaucke@redhat.com>
or it may not be deleted, plus a spell fix

Signed-off-by: Jakob Naucke <jnaucke@redhat.com>
@Jakob-Naucke

Copy link
Copy Markdown
Member Author

we got #285 merged and this is rebased (10 -> 5 commits)

Comment thread test_utils/src/virt/kubevirt.rs Outdated
@Jakob-Naucke

Copy link
Copy Markdown
Member Author

control plane didn't come up on the CI that this PR intends to replace

/retest

Also tests fine with 500m, which aligns better with how much compute
we get on GHA.

Signed-off-by: Jakob Naucke <jnaucke@redhat.com>
avoiding maintenance and lack of parallelization on a CI host &
scripting logic in openshift-ci

Signed-off-by: Jakob Naucke <jnaucke@redhat.com>
Assisted-by: Opus 4.6
@openshift-ci

openshift-ci Bot commented Jun 26, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: alicefr, Jakob-Naucke

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Jakob-Naucke

Jakob-Naucke commented Jun 26, 2026

Copy link
Copy Markdown
Member Author

thanks for your review @alicefr. let's wait this run again and if it succeeds, I'll merge and raise the removal in openshift-release (-3kloc)

@Jakob-Naucke Jakob-Naucke merged commit de12968 into trusted-execution-clusters:main Jun 26, 2026
11 checks passed
@Jakob-Naucke Jakob-Naucke deleted the gha-kubevirt branch June 26, 2026 15:01
Jakob-Naucke added a commit to Jakob-Naucke/openshift-release that referenced this pull request Jun 29, 2026
Test relied on moving PR code to another host and running KinD there,
which has been superseded by running on GHA
trusted-execution-clusters/operator#287.

Keeping basic structure for trusted-execution-clusters because we
still intend to utilize OpenShift on openshift#79393.

Signed-off-by: Jakob Naucke <jnaucke@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants